C ONSTRAINT - B ASED M INING AND L EARNING AT ECML / PKDD 2007 CMILE ’ 07 September 21 , 2007 Warsaw , Poland

Machine learning techniques are increasingly being used to produce a wide-range of classifiers for complex real-world applications that involve different constraints both on the resources allocated for the learning process and on the resources used by the induced model for future classification. As the complexity of these applications grows, the management of these resources becomes a challenging task. In this work we introduce ACT (Anytime Cost-sensitive Tree learner), a novel framework for operating in such environments. ACT is an anytime algorithm that allows trading computation time for lower classification costs. It builds a tree top-down and exploits additional time resources to obtain better estimations for the utility of the different candidate splits. Using sampling techniques ACT approximates for each candidate split the utility of the subtree under it and favors a split with the best evaluation. Due to its stochastic nature ACT is expected to be able to escape local minima, into which greedy methods may be trapped. ACT can be applied in two anytime setups: the contract setup where the allocation of resources is known in advance, and the interruptible setup where the algorithm might be queried for a solution at any point of time. Experiments with a variety of datasets were conducted to compare the performance of ACT to that of the state of the art decision tree learners. The results show that for most domains ACT produces significantly better trees. In the cost-insensitive setup, where test costs are ignored, ACT could produce smaller and more accurate trees. When test costs are involved, the ACT trees were significantly more efficient for classification. ACT is also shown to exhibit good anytime behavior with diminishing returns. S. Esmeir and S. Markovitch. Anytime Learning of Decision Trees. Journal of Machine Learning Research (JMLR), 8, 2007. S. Esmeir and S. Markovitch. Occam’s Razor Just Got Sharper. In Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI-2007), 2007. S. Esmeir and S. Markovitch. Anytime Induction of Decision Trees: An Iterative Improvement Approach. In Proceedings of The 21st National Conference on Artificial Intelligence (AAAI2006), 2006. S. Esmeir and S. Markovitch. When a Decision Tree Learner Has Plenty of Time. In Proceedings of The 21st National Conference on Artificial Intelligence (AAAI-2006), 2006. S. Esmeir and S. Markovitch. Lookahead-based Algorithms for Anytime Induction of Decision Trees. In Proceedings of the 21st International Conference on Machine Learning (ICML-2004), 2004. Constraint Based Hierarchical Clustering for Text Documents